Text input constitutes one of the most frequent computer user tasks. The QWERTY keyboard has been accepted as the standard tool for text entry for desktop computing. However, the emergence of handheld and other forms of pervasive or mobile computing calls for alternative solutions. These devices have small screens and limited keypads, limiting the ability of the user to input text. Consequently, text input has been revived as a critical research topic in recent years. The two classes of solutions that have attracted the most attention are handwriting and stylus-based virtual keyboarding.
Handwriting is a rather “natural” and fluid mode of text entry due to users' prior experience. Various handwriting recognition systems have been used in commercial products. However, the fundamental weakness of handwriting as a text entry method is its limited speed. While adequate for entering names and phone numbers, handwriting is too limited for writing longer text.
Virtual keyboards tapped serially with a stylus are also available in commercial products. The keyboard provided on the screen is typically the familiar QWERTY layout. Stylus keyboarding requires intense visual attention at almost every key tap, preventing the user from focusing attention on text output. To improve movement efficiency, optimization of the stylus keyboard layout has been considered both by trial and error and algorithmically. Using a keyboard layout such as ATOMIK (Alphabetically Tuned and Optimized Mobile Interface Keyboard), text entry is relatively fast. Reference is made to S. Zhai, M. Hunter & B. A. Smith, “Performance Optimization of Virtual Keyboards, Human-Computer Interaction,” Vol.17 (2, 3), 229–270, 2002.
The need for entering text on mobile devices has driven numerous inventions in text entry in recent years. The idea of optimizing gesture for speed is embodied in the Unistrokes alphabet. In the Unistrokes alphabet, every letter is written with a single stroke but the more frequent ones are assigned simpler strokes. If mastered, a user could potentially write faster in the Unistrokes alphabet than in the Roman alphabet. The fundamental limitation of the Unistrokes alphabet, however, is the nature of writing one letter at a time.
Quikwriting method uses continuous stylus movement on a radial layout to enter letters. Each character is entered by moving the stylus from the center of the radial layout to one of the eight outer zones, sometimes crossing to another zone, and returning to the center zone. The stylus trajectory determines which letter is selected. While it is possible to develop “iconic gestures” for common words like “the”, such gestures are relatively complex due to the fact that the stylus has to return to the center after every letter. In this sense, the Quikwriting method is fundamentally a character entry method.
Cirrin (Circular Input) operates on letters laid out on a circle. The user draws a word by moving the stylus through the letters. Cirrin explicitly attempts to operate on a word level, with the pen being lifted up at the end of each word. Cirrin also attempts to optimize pen movement by arranging the most common letters closer to each other. However, Cirrin is neither location nor scale independent.
It is important to achieve at least partial scale and location independency for the ease and speed of text entry. If all the letters defining a word on the keyboard have to be precisely crossed, the time to trace these patterns cannot be expected to be any shorter than tapping. As an example, if it is desired to draw a line from key “r” to key “d” as part of the word “word”, within a tunnel connecting the two keys, such a closed-loop drawing process would take more time and visual attention than tapping “d” after “r”. The user must place the pen in the proper position before drawing the word and ensure that the movement of the pen from letter to letter falls within the allowed pen stroke boundaries.
It is also important to facilitate skill transfer from novice behavior to expert performance in text entry by designing similar movement patterns for both types of behavior. The idea of bridging novice and expert modes of use by common movement pattern is used in the “marking menu”. Instead of having pull-down menus and shortcut keys, two distinct modes of operation for novice and expert users respectively, a marking menu uses the same directional gesture on a pie menu for both types of users. For a novice user whose action is slow and needs visual guidance, marking menu “reveals” itself by displaying the menu layout after a pre-set time delay. For an expert user whose action is fast, the marking menu system does not display visual guidance. Consequently, the user's actions become open loop marks. However, the marking menu is not used for text entry due to the limited number of items can be reliably used in each level of a pie menu (8 or at the most 12). Reference is made to G. Kurtenbach, and W. Buxton, “User Learning and Performance with Marking Menus,” Proc. CHI. 1994, pages 258–264; and G. Kurtenbach, A. Sellen, and W. Buxton, “An Empirical Evaluation of Some Articulatory and Cognitive Aspects of “Marking Menus”, ”Human Computer Interaction, 1993, 8(1), pages 1–23.
A self-revealing menu approach, T-Cube, defines an alphabet set by cascaded pie menus that are similar to a marking menu. A novice user enters characters by following the visual guidance of menus, while an expert user could enter the individual characters by making menu gestures without visual display. A weakness of the T-Cube is that it works at alphabet level; consequently, text entry using T-cube is inherently slow.
Dasher, another approach using continuous gesture input, dynamically arranges letters in multiple columns. Based on preceding context, likely target letters appear closer to the user's cursor location. A letter is selected when it passes through the cursor; consequently, cursor movement is minimized. This minimization, however, is at the expense of visual attention. Because the letter arrangement constantly changes, Dasher demands user's visual attention to dynamically react to the changing layout.
One possibility for introducing gesture-based text entry would be the use of shorthand. Traditional shorthand systems are efficient, but hard to learn for the user and difficult to recognize by the computer. Shorthand has no duality; it cannot be used by experts and beginners alike. In addition, shorthand has no basis in a virtual keyboard, so the user cannot identify the required symbol from the keyboard. If the user forgets shorthand symbols, a separate table must be consulted to find the symbol.
What is therefore needed is a form of continuous gesture-based text input that requires minimal visual attention, and that is based on keyboard entry, wherein a system and method recognize word patterns based on a virtual keyboard layout. The need for such system and method has heretofore remained unsatisfied.